Tiny refactor onSchemaCreate function.

…ble units ("100 GiB") (pingcap#471)

* Makefile: add `make finish-prepare` action

* config: accept human-readable size for most byte-related config

e.g. allow `region-split-size = '96M'` in additional to `= 100663296`

(known issue: these values' precisions will be truncated to 53 bits
instead of supporting all 63 bits)

* restore: reduce chance of spurious errors from TestGcTTLManagerSingle

Co-authored-by: glorv <[email protected]>

* save my work

add notes

save work

save work

fix unit test

remove tidbMgr in RestoreController

remove some comments

remove some comments

change logger in SQLWithRetry

revert replace log.Logger to *zap.Logger

dep: update uuid dependency to latest google/uuid (pingcap#452)

* dep: update satori/go.uuid to latest

* fix tests

* change to google/uuid

* fix build

* try fix test

* get familiar with google/uuid

* address comment

tidb-lightning-ctl: change default of -d to 'noop://' (pingcap#453)

also add noop:// to supported storage types (to represent an empty store)

replace tab to space

try another port to fix CI

remove some comment

*: more glue

restore: fix the bug that gc life time ttl does not take effect (pingcap#448)

* fix gc ttl loop

* resolve comment and add tests

fix CI

report info to host TiDB

config: filter out all system schemas by default (pingcap#459)

backend: fix auto random default value for primary key (pingcap#457)

* fix auto generate auto random primary key column

* fix default for auto random primary key

* fix test

* use prev row id for auto random and add a test

* replace chunck with session opt

* fix

* fix

mydumper: fix parquet data parser (pingcap#435)

* fix parquet

* reorder imports

* fix test

* use empty collation

* fix a error and add more test cases

* add pointer type tests

* resolve comments

Co-authored-by: kennytm <[email protected]>

address comment

backend/local: use range properties to optimize region range estimate (pingcap#422)

* use range propreties to estimate region range

* post-restore: add optional level for post-restore operations (pingcap#421)

* add optional level for opst-restore operations

* trim leading and suffix '"

* use UnmarshalTOML to unmarshal post restore op level

* resolve comments and fix unit test

* backend/local: do not retry epochNotMatch error when ingest sst (pingcap#419)

* do not retry epochNotMatch error when ingest sst

* add retry ingest for 'Raft raft: proposal dropped' error in ingest

* change some retryable error log level from Error to Warn

* fix nextKey

* add a comment for nextKey

* fix comment and add a unit test

* wrap time.Sleep in select

Co-authored-by: kennytm <[email protected]>

* update

* use range properties to optimze region range estimate

* update pebble

* change the default value for batch-size

* add unit tests and reslove comments

* add a comment to range properties test

* add a comment

* add a test for range property with pebble

* rename const variable

Co-authored-by: kennytm <[email protected]>

fix pd service id is empty (pingcap#460)

fix s3 parquet reader (pingcap#461)

Co-authored-by: Neil Shen <[email protected]>

fix service gc ttl again (pingcap#465)

address comment

mydumper: verify file routing config (pingcap#470)

* fix file routing

* remove useless line

* remove redundant if check

rename a method in interface

save work

try fix CI

could work

change ctx usage

try fix CI

try fix CI

refine function interface

refine some fucntion interface

debug CI

address comment

config: allow four byte-size config to be specified using human-readable units ("100 GiB") (pingcap#471)

* Makefile: add `make finish-prepare` action

* config: accept human-readable size for most byte-related config

e.g. allow `region-split-size = '96M'` in additional to `= 100663296`

(known issue: these values' precisions will be truncated to 53 bits
instead of supporting all 63 bits)

* restore: reduce chance of spurious errors from TestGcTTLManagerSingle

Co-authored-by: glorv <[email protected]>

remove debug log

test: change double type syntax (pingcap#474)

address comment

checkpoint: add glue checkpoint

resolve cycle import

expose Retry

refine

change interface to cope with TiDB

fix SQL string

fix SQL

adjust interface to embedded in TiDB

could import now

reduce TLS

restore: add `glue.Glue` interface and other function (pingcap#456)

* save my work

* add notes

* save work

* save work

* fix unit test

* remove tidbMgr in RestoreController

* remove some comments

* remove some comments

* change logger in SQLWithRetry

* revert replace log.Logger to *zap.Logger

* replace tab to space

* try another port to fix CI

* remove some comment

* *: more glue

* report info to host TiDB

* fix CI

* address comment

* address comment

* rename a method in interface

* save work

* try fix CI

* could work

* change ctx usage

* try fix CI

* try fix CI

* refine function interface

* refine some fucntion interface

* debug CI

* address comment

* remove debug log

* address comment

modify code

add comment

refine some code

* address comment

* add some comments

* fix CI and change CREATE TABLE

…Y` syntax (pingcap#471)

…Y` syntax (pingcap#471)

…Y` syntax (#471)

Signed-off-by: AmoebaProtozoa <[email protected]>

* Adds a feature whitelist at planbuild stage (pingcap#304)

* init

Signed-off-by: David <[email protected]>

* format errors

Signed-off-by: David <[email protected]>

* restrict information schema

Signed-off-by: David <[email protected]>

* fmt

Signed-off-by: David <[email protected]>

* bazel

Signed-off-by: David <[email protected]>

* fmt

Signed-off-by: David <[email protected]>

* disable datadir

Signed-off-by: David <[email protected]>

* add ttl check for create and alter table

Signed-off-by: David <[email protected]>

* create table ttl

Signed-off-by: David <[email protected]>

* address comments

Signed-off-by: David <[email protected]>

* update error doc

Signed-off-by: David <[email protected]>

* add the back to the error message

Signed-off-by: David <[email protected]>

* unblock 'SHOW CONFIG' for lightning

Signed-off-by: David <[email protected]>

---------

Signed-off-by: David <[email protected]>
Co-authored-by: zzm <[email protected]>
Signed-off-by: AmoebaProtozoa <[email protected]>
# Conflicts:
#	ddl/BUILD.bazel
#	errno/errname.go
#	util/sem/sem.go

* add test flag that can disable feature control (pingcap#328)

Signed-off-by: David <[email protected]>
Co-authored-by: iosmanthus <[email protected]>

* Adds a feature whitelist at planbuild stage (pingcap#304)

* init

Signed-off-by: David <[email protected]>

* format errors

Signed-off-by: David <[email protected]>

* restrict information schema

Signed-off-by: David <[email protected]>

* fmt

Signed-off-by: David <[email protected]>

* bazel

Signed-off-by: David <[email protected]>

* fmt

Signed-off-by: David <[email protected]>

* disable datadir

Signed-off-by: David <[email protected]>

* add ttl check for create and alter table

Signed-off-by: David <[email protected]>

* create table ttl

Signed-off-by: David <[email protected]>

* address comments

Signed-off-by: David <[email protected]>

* update error doc

Signed-off-by: David <[email protected]>

* add the back to the error message

Signed-off-by: David <[email protected]>

* unblock 'SHOW CONFIG' for lightning

Signed-off-by: David <[email protected]>

---------

Signed-off-by: David <[email protected]>
Co-authored-by: zzm <[email protected]>
Signed-off-by: AmoebaProtozoa <[email protected]>
# Conflicts:
#	ddl/BUILD.bazel
#	errno/errname.go
#	util/sem/sem.go

* disable validate password (pingcap#465)

Signed-off-by: AmoebaProtozoa <[email protected]>

* turn on validate password and set special char count to 0 (pingcap#464)

* serverless version 16

Signed-off-by: AmoebaProtozoa <[email protected]>

* fix typo in comments

Signed-off-by: AmoebaProtozoa <[email protected]>

* set special char to 0

Signed-off-by: AmoebaProtozoa <[email protected]>

---------

Signed-off-by: AmoebaProtozoa <[email protected]>

* disabling password validation (pingcap#467)

* skip enabling password validation

Signed-off-by: AmoebaProtozoa <[email protected]>

* fix global variable

Signed-off-by: AmoebaProtozoa <[email protected]>

---------

Signed-off-by: AmoebaProtozoa <[email protected]>

* allow edit waittimeout (pingcap#468)

Signed-off-by: AmoebaProtozoa <[email protected]>

* allow edit  tidb-isolation-read-engines (pingcap#470)

Signed-off-by: AmoebaProtozoa <[email protected]>

* allow edit tidb_txn_mode (pingcap#471)

Signed-off-by: AmoebaProtozoa <[email protected]>

* allow edit tidb-cdc-write-source (pingcap#473)

Signed-off-by: AmoebaProtozoa <[email protected]>

* allow edit tidb_placement_mode (pingcap#474)

Signed-off-by: AmoebaProtozoa <[email protected]>

* please the checker

Signed-off-by: AmoebaProtozoa <[email protected]>

* sem enable now returns error on fail

Signed-off-by: AmoebaProtozoa <[email protected]>

* make check

Signed-off-by: AmoebaProtozoa <[email protected]>

---------

Signed-off-by: David <[email protected]>
Signed-off-by: AmoebaProtozoa <[email protected]>
Co-authored-by: iosmanthus <[email protected]>
Co-authored-by: zzm <[email protected]>